Knowledge Discovery and Data Mining in Databases

Knowledge Discovery in Databases (KDD) is the process of automatic discovery of previously unknown patterns, rules, and other regular contents implicitly present in large volumes of data. Data Mining (DM) denotes discovery of patterns in a data set previously prepared in a specific way. DM is often used as a synonym for KDD. However, strictly speaking DM is just a central phase of the entire process of KDD. The purpose of this chapter is to gradually introduce the process of KDD and typical DM tasks. The idea of automatic knowledge discovery in large databases is first presented informally, by describing some practical needs of users of modern database systems. Several important concepts are then formally defined and the typical context and resources for KDD are discussed. Then the scope of KDD and DM is briefly presented in terms of classification of KDD/DM problems and common points between KDD and several other scientific and technical disciplines that have well-developed methodologies and techniques used in the field of KDD. After that, the chapter describes the typical KDD process, DM tasks and some algorithms that are most frequently used to carry out such tasks. Some other important aspects of KDD are covered as well, such as using domain knowledge in the KDD process and evaluating discovered patterns. Finally, the chapter briefly surveys some important KDD application domains and practical KDD/DM systems, and discusses several hot topics and research problems in the field that are of interest to software industry.